Audiovisual display modes for sense-and-avoid system for aerial vehicles

ABSTRACT

The invention provides six different display modes illustrating interaction and relative locations of two or more aerial vehicles (AVs), with at least one of the AVs being controllable by a ground-based or airborne-based controller of an unmanned aerial vehicle (UAV) or a pilot of a standard manned aircraft. Some display modes also indicate a predicted distance of closest approach of two AVs, the possibility of conflict or collision, and a remaining time, measured relative to the present time, before this conflict occurs. An audio and/or visual indicator advises the AV controller if this conflict event is likely to occur and recommends an acceleration or deceleration increment that may avoid such conflict.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of application Ser. No. 13/694,940,filed Jan. 22, 2013, which is a continuation-in-part of application Ser.No. 12/661,672, filed Mar. 22, 2010, now U.S. Pat. No. 8,370,057, whichis a continuation-in-part of application Ser. No. 11/888,070, filed Jul.31, 2007, now U.S. Pat. No. 7,706,979, which is a continuation-in-partof application Ser. No. 11/120,263, filed May 3, 2005, now U.S. Pat. No.7,269,513. Each of the foregoing applications is hereby incorporated byreference in its entirety.

FIELD OF THE INVENTION

This invention relates to computer screen display modes for two or moreaerial vehicles (“AVs”), including an unmanned aerial vehicle (“UAVs”),using a ground-based or airborne-based sense-and-avoid display systemfor monitoring flight safety and navigation. The sense-and-avoid displaysystem, hereafter referred to as the “system,” provides the AVcontroller with an audiovisual (i.e., audio and/or visual) presentationof information pertaining to airborne conflict detection and collisionavoidance. The AV controller may be a ground-based or airborne-based UAVcontroller or a cockpit-based pilot of a standard manned aircraft.

The novelties disclosed herein follow from the original invention in theParent Application (U.S. Pat. No. 7,269,513, issued Sep. 11, 2007) andthe first Continuation in Part (CIP) (U.S. Ser. No. 11/888,070, filedJul. 31, 2007). The essence of the inventions set forth in the ParentApplication and the first CIP remain essentially the same.

BACKGROUND OF THE INVENTION

Flight safety is a key factor associated with the implementation of UAVflight operations in the National Airspace System (NAS). For flightsafety, the determination and display of the flight paths of two or moreaerial vehicles (“AVs”) in the same three dimensional (3D) airspacerequires estimating and displaying the time and distance of closestapproach between two AVs, where one of the AVs may be aground-controlled or airborne-controlled UAV. One of the challenges isdisplaying the computed closest approach information in a readilyunderstandable and timely manner for UAV controllers operating underVisual Flight Rules (VFR).

When operating a manned aircraft under VFR in the NAS, visual referenceto the airspace outside the cockpit is the regulation under which pilotscontrol their aircraft's altitude and flight path. The fundamentalflight safety principle of VFR is that the pilots of manned aircrafthave the responsibility of maintaining safe separation from other AVs.For UAV flight operations under VFR, it is logical to assume that UAVcontrollers need to be equipped with a system that provides anequivalent or better level of safety compared with the visual capabilityof pilots in manned aircraft. The system needs to locate and track otherAVs, including those AVs not equipped with an identification device(e.g., a transponder), at a sufficient range in order to detectpotential airborne conflicts and maintain safe separation distances.

The critical steps for actual safe separation and collision avoidance issubdivided into two categories: (i) computations and correspondingrecommendations provided by the system; and (ii) tasks performed by aUAV controller. The UAV controller is equivalent to an operator or pilotresponsible for changing the flight vector and flight speed of the UAVfor the purpose of collision avoidance and the maintenance of safeseparation distances from other AVs.

The preceding patent applications have disclosed some approaches foranalyzing situations involving two or more aerial vehicles that areairborne in the vicinity of each other, for navigation and flight safetypurposes. It would be useful to provide screen displays that moredirectly illustrate the present situation and a prediction of at leastone scenario (e.g., a speculative “worst case analysis”) that maydevelop, if the present situation continues. Preferably, a UAVcontroller or AV pilot or other observer should be given a choice amongtwo or more available audiovisual display modes that can be rapidlyswitched between, depending upon the observer's preference and upon thesituation parameters. Preferably, a display mode should provide graphicdata substantially in real time, with an associated latency no greaterthan a time interval between two consecutive computer screen refreshesat rates chosen by the UAV controller or AV pilot (as quickly as 2seconds in one approach).

What is needed is a sense-and-avoid display system that provides the UAVcontroller or AV pilot with a graphic display of air traffic activity ofa selected AV and its relation to one or more adjacent AVs. The systemneeds to highlight potential airborne conflict, including but notlimited to collisions and provision of user-defined warning zonesinvolving the UAV, and provide recommendations for maintaining a safeseparation distance between the UAV and other AVs in real time ornear-real time.

SUMMARY OF THE INVENTION

These needs are met by the invention, which provides several differentdisplay modes for present locations and predicted future locations, foreach of two or more airborne aerial vehicles AV1 and AV2 that operate inthe same region. These needs are met by showing potential conflictsbetween the two AVs where at least one of the AVs is a UAV. At least oneof the AVs (e.g., the first AV) should be a controllable UAV. Because ofthe two dimensional nature of a graphic display screen, each of thedisplay modes has associated strengths and deficits that may becompensated in another display mode.

In a first display mode, the present location of each of two or more AVsis projected onto, and displayed on: (1) a first, vertically orienteddisplay plane Π(EW), which is defined by a local z-axis direction(vertical) and by an local x-axis, orthogonal to the z-axis, extendingin an east-west direction; and (2) a second, vertically oriented displayplane Π(NS), which is defined by a local z-axis direction and by a localy-axis, orthogonal to the z-axis and to the x-axis, extending in anorth-south direction. Each of the planes, Π(EW) and Π(NS), serves as adisplay plane Π(1), or these two planes can serve as a combined displayplane. The planes Π(EW) and Π(NS) intersect, and the first AV presentlocation vector, r10=r1(t0), lies on a line segment of intersection ofΠ(EW) and Π(NS), where t0 is the present time. The two planes Π(EW) andΠ(NS) are preferably viewed in a perspective view, in this first displaymode. A present location, r20=r2(t0), of a second AV is projected,parallel to the x-axis and parallel to the y-axis onto the respectiveplanes Π(EW) and Π(NS), using a formalism developed in Appendix A, or anequivalent formalism. The projections of the location r20 on the planesΠ(EW) and Π(NS) will vary with the changing location of the second AVrelative to the location of the first AV.

Each of the display screens in each of the display modes (1-6)optionally includes a supplemental first scale S1 that graphicallyprovides (1) a visually perceptible first length L1 that is linearlyproportional to a present separation distance |r10-r20| of the first andsecond AVs and optionally provides a supplemental second scale S2 thatgraphically provides a second length L2 that is linearly proportional toa closing rate value, CRV=(∂/∂t) r1(t)−r2(t)| at a chosen value of timet, such as the present time, t=t0, and indicate with two opposed arrowswhether the CRV>0 (arrows point away from each other) or the CRV<0(arrows point toward each other, indicating AVs approach each other).

In a second display mode, a third, vertically oriented display planeΠ(φ;1,2) is defined by a local z-axis direction and by an localazimuthal axis, orthogonal to the z-axis and oriented at an azimuthalangle φ relative to the local x-axis. An anchor point for the displayplane Π(2)=Π(φ;1,2) and the azimuthal angle φ are chosen so that thisazimuthal plane passes through the present locations, r1(t) and r2(t),of each of AV1 and AV2. The plane Π(φ;1,2), is uniquely defined unlessthe present locations, r1(t0) and r2(t0), coincide. Where N AVs arepresent, numbered n=1, 2, . . . , N (N≧3), an azimuthal plane Π(φ;n1,n2)can be determined and displayed separately for each two AVs of interest,for example, two AVs (n=n1 and n=n2) that presently have the smallestseparation distance |r10-r20|, where, at least, one of the AVs is a UAV.Optionally, a plane Π(φ; n1,n2) can be displayed for two AVs (n=n1 andn=n2), containing the locations (vectors) rn1(t) and rn1(t), for which|rn10−rn20| is the smallest for the present time t.

A third display mode provides a nadir (overhead) view of the first andsecond AVs. The anchor point for this horizontal display plane Π(3)preferably passes through the present location of the first AV (z=z1),or through the present location of the second AV (z=2), or through avertical location intermediate between the first and second AVs(z=f·z1+(1−f)·z2, with 0<f<1; for example, f=0.5). The locations of thefirst and second AVs are projected vertically onto Π(3), and eachprojection has an associated arrow of length L3 proportional to theclosing rate value CRV=(∂/∂t)|r1(t)−r2(t)|, where the two arrows pointtoward each other if CRV<0 and point away from each other if CRV>0.Optionally, a supplemental scale of length L3′, located adjacent to aboundary of the display plane Π(3), indicates vertical separationΔz=|z1−z2| of the first and second AVs.

In a fourth display mode, an initial separation vectorΔr12(t0)=r1(t0)−r2(t0) and a separation distance squared,d(t;1,2)²=|Δr12(t)|² are determined for first and second AVs, usingobserved or estimated vector values for the present location rj(t0)(j=1, 2), the present velocity vector vj0=vj(t0) and the presentacceleration vector aj0=aj(t0) for each AV. A prediction of a futureseparation vector is estimated, using r10-r20, v10-v20, and a10-a20, andan analysis disclosed in a preceding patent application, U.S. Ser. No.11/888,070 (incorporated by reference herein), and one or more times,t=t(min), are determined for which d(t;1,2)²=|Δr12(t)|² is minimized.One or three real solutions, t=t(min) can be found, and interest centerson a first real solution for which t(min)≧t0.

A unit length normal n(4)=(v10̂a10)/|v10̂a10| is determined, and a displayplane Π(4) is identified, with normal vector n(4) and anchor point givenby the location r10. Where the vectors v10 and a10 are substantiallyparallel, a vertically oriented plane Π′(4), generated by the vectors kand v10, becomes the replaces the plane Π(4), with corresponding normalvector n′(4)={k̂v10}/|k̂v10|. The location r20 of the second AV is shownrelative to the display plane Π(4). A predicted AV trajectory r1(t)(t0(fixed)≦t≦t(min)) for the first AV lies in, and does not deviatefrom, the display plane Π(4), and intermediate locations r1(t′) for thefirst AV trajectory are shown on the display plane Π(4). The systemoptionally indicates the value d(t=t(min);1,2) and determines whetherthis minimum separation distance satisfies d(t=t(min);1,2)≦r(thr0),where r(thr0) is a selected conflict radius of a sphere Sph(4), centeredat the location r1(t), and the separation vector Δr12(t) should avoidthe sphere interior. The interior may be a region where a collision ofAV1 and AV2 may occur, or where these two AVs may experience a nearmiss, for example, r(thr0)=100-1320 feet.

Where d(t=t(min);1,2)>r(thr0), the separation distance d(t;1,2) for thepredicted trajectories of AVs number 1 and 2 will always be greater thanr(thr0). In this situation, no further graphics are required for thefourth display mode. Where d(t(min);1,2)≦r(thr0), the system (1)estimates a first time, t=t1, that satisfies t0≦t1≦t(min) andd(t=t1;1,2)=r(thr0), (2) determines a location r1(t1), (3) displays acircle Cir(4) in the display plane Π(4), centered at r1(t1), of radiusr(thr0) (representing a predicted conflict sphere), (4) determines aremaining time, Δt(rem)=t1-t0, before conflict will occur, and (5)provides a visually perceptible and/or audibly perceptible signalindicating that a conflict will occur and the remaining time Δt(rem).This situation is illustrated in FIG. 4. Optionally, the systemrecommends an acceleration increment, Δa1(t0) that AV1 should execute inorder to avoid the predicted conflict.

In a fifth display mode, a display plane Π(5) is defined by the presentvelocity vectors, v10 and v20, of the first and second AVs, with a unitlength normal vector n(5)={V10̂v20}/|v10̂v20|. The display plane Π(5)contains the present location r10, and contains a projection of thesecond AV r20, parallel to the normal vector n(5), onto the plane Π(5),determined as disclosed in Appendix A.

In a sixth display mode, the display plane Π(6) is defined by thepresent vectors, v10 and a10, with a unit length normal vectorn(6)={v10̂a10}/|v10̂a10| and containing the present location r10, Wherethe vectors v10 and a10 are substantially parallel, a verticallyoriented plane, generated by the vectors k and v10, becomes a planeΠ′(6) that replaces the plane ∪(6), with corresponding normal vectorn′(6)={k̂v10}/|k̂v10|. The present location r20 of the second AV isprojected parallel to the normal vector n(6) onto the plane Π(6).Trajectories, r1(t) and r2(t), are optionally shown for AV1 and AV2 ateach of a sequence of two or more spaced apart times (e.g., as dotted ordashed lines), from the present time value, t=t0, to a point of closestapproach, t=t(min), for the AVs, which will generally not lie on theplane Π(6).

The novelties of this invention include a variety of audiovisual displaymodes driven by the need for clarity of presentation and ease ofinterpretation by UAV controllers or AV pilots. In the context of UAVflight operations, the human factor is recognized as a key element forevaluating a UAV controller's capability to maintain safe separationdistances from other AVs at a level of safety equivalent to or betterthan the cockpit-based eyes of pilots in manned aircraft. Theaudiovisual display modes of the system enable the UAV controller toclearly understand potential airborne conflicts during UAV flightoperations, including flights that extend beyond the visual range ofground-based observers and airborne observers in chase planes.

BRIEF DESCRIPTION OF THE FIGURES

FIGS. 1 and 2 illustrate orientation of the planes Π(EW) and Π(NS) andΠ(φ), for the first and second display modes.

FIGS. 3A and 3B illustrate a nadir view presented in a third displaymode.

FIG. 4 illustrates a minimum separation display determined in a fourthdisplay mode.

FIGS. 5A and 5B illustrate two alternatives situations in which displaymode 4 can be presented.

FIGS. 6 and 7 illustrate display configurations used in the fifth andsixth display modes.

FIGS. 8 and 9 illustrate some geometric determinations used in variousembodiments.

FIG. 10 illustrates a procedure for determination of an accelerationincrement to avoid conflict of collision.

FIGS. 11 and 12 and 13 illustrate alternative screen displays, includinga primary mode and one or more secondary modes, according to theinvention.

DESCRIPTION OF BEST MODES OF THE INVENTION

The invention provides several different screen display modes for aground-based or airborne controller (collectively referred to as a“controller”) of an aerial vehicle (AV). Two or more (or all) modes canbe displayed simultaneously, or the controller can switch from onedisplay mode to another depending upon the circumstances and upon whichmode(s) are more relevant, as illustrated in FIGS. 11 and 12 and 13.Each of the Figures has an associated Cartesian coordinate system(x,y,z), with corresponding unit length vectors (i, j, k), in which thez-axis corresponds to a local vertical direction. Each of the followingdisplay modes displays a portion of one or more display planes, as thisplane would appear on a screen observed by a controller. The x-axis andy-axes define a locally horizontal plane, and are orthogonal to eachother and to the z-axis.

In a first display mode, illustrated in FIG. 1, the present location ofeach of two or more AVs is projected onto, and displayed on, (1) afirst, vertically oriented display plane Π(EW), which is defined by alocal z-axis (vertical) direction and by an local x-axis, orthogonal tothe z-axis and extending in an east-west direction; and/or (2) a second,vertically oriented display plane Π(NS), which is defined by the localz-axis direction and by a local y-axis, orthogonal to the z-axis andextending in a north-south direction. The planes Π(EW) and Π(NS)intersect at right angles, and the first AV present location r10=r1(t0)is on a line segment of intersection of these planes, corresponding tothe altitude, z=z10, of the present location of the first AV. The secondAV present location r20=r21(t0) will usually be located at a point offone or both of the planes, Π(EW) and Π(NS). Preferably, these two planesare viewed together in a perspective view, relative to the respectiveplanes Π(EW) and Π(NS), as illustrated in FIG. 1. The present locationr20 of the second AV is projected, parallel to the y-axis and parallelto the x-axis onto the planes Π(EW) and Π(NS), respectively, using aformalism developed in Appendix A The projections of the presentlocation r20 of the second AV on the planes Π(EW) and Π(NS) will varywith the changing location r20 of the second AV relative to the locationr10 of the first AV.

The vertically oriented plane ∪(EW) is a plane parallel to the x-axisand parallel to the z-axis and is defined by a coordinate relation

y=y10(constant),   (1A)

with a corresponding unit length normal vector

n=j.   (1B)

The vertically oriented plane Π(NS), parallel to the y-axis and to thez-axis, is similarly defined by a coordinate relation

x=x10(constant).   (2A)

with a corresponding unit length normal vector

n=i.   (2B)

r10=(x10,y10,z10)   (3)

An anchor point for an intersection of the vertical planes, Π(EW) andΠ(NS), has the coordinates r10=(x10,y10,z10), where z10 is the zcoordinate of the first AV, and r20 is projected perpendicular onto eachof the planes Π(EW) and Π(NS), as illustrated in FIGS. 1 and 9. Theplanes Π(EW) and Π(NS) can be used individually as display planes, withone or both of the present locations r10 and/or r20 being projected ontothe display plane, computed as follows:

r(proj)=r−{(r−r(i))·n}n. (r=r10 or r20).   (4)

where r(i) is a vector, partly coinciding with the vector r, that pointsto the intersection of R with a display plane that is defined by n.

Preferably, the planes Π(EW) and Π(NS) are shown together at anon-horizontal, perspective viewing direction, as illustrated in FIG. 1,with the present location r10 being on an intersection line of Π(EW) andΠ(NS), and the location r20 being projected perpendicularly onto Π(EW)and onto Π(NS), as shown in FIG. 1.

In a second display mode, a third, vertically oriented display planeΠ(2)=Π(φ;1,2), illustrated in FIG. 2, is defined by a local z-axisdirection and by an local azimuthal axis, orthogonal to the z-axis andoriented at an azimuthal angle φ relative to the local x-axis, asillustrated in FIG. 2 An anchor point for the display plane Π2=Π(φ;1,2)and the azimuthal angle φ are chosen so that this plane passes throughthe present locations, r10 and r20, of both AV1 and AV2. The planeΠ(φ;1,2), is uniquely defined unless the present locations, r1(t0) andr2(t0), coincide with each other or with the z-axis.

Optionally, first and second arrows extend from the present first andsecond AV locations, r10 and r20, where the two arrows (1) point towardeach other when a closing rate value CRV=(∂/∂t) Irl (t)|r1<0, and (2)the two arrows point away from each other when a closing rate valueCRV=(∂/∂t)|r1(t)−r2(t)|>0. A single arrow, rather than two such arrows,can be used here. Where N AVs are present, numbered n=1, 2, . . . , N(N≧3), the plane Π(φ;n1,n2) can be determined and displayed separatelyfor each two AVs (n1 and n2) of interest. Optionally, a plane Π(φ;n1,n2)can be displayed for two AVs containing the locations rn1(t0) andrn2(t0), for which |rn1(t0)−rn2(t0)| is the smallest for the presenttime, t=t0.

The vertically oriented plane Π(φ;1,2) of the second mode, illustratedin FIG. 2, is defined by a relation

(x−x0′)sin φ−(y−y0′)cos φ=0,   (5A)

with corresponding unit length normal vector

n=−i sin φ+j cos φ,   (5B)

where φ is the selected azimuthal angle (0≦φ<2π) and an anchor point,(x=x0′,y=y0′, z=z0′), is unspecified. An anchor point for the planeΠ(φ;1,2), is a location (x0′ ,y0′ ,z0′) in three dimensions throughwhich the plane Π(φ;1,2) passes, A (three dimensional) anchor point andan azimuthal angle φ for the plane Π(φ) are chosen so that this planeΠ(φ;1,2) includes each of the AV present locations r10=(x10,y10,z10) andr20=(x20,y20,z20). This requires that

tan φ=(y20−y10)/(x20−x10) (x20−x10≠0),   (6)

(x10−x0′)sin φ−(y1−y0′)cos φ=0,   (7A)

(_i x2−x0′)sin φ−(y2−y0′)cos φ=0,   (7B)

and is satisfied, for example, by the choice

(x0′,y0′)=((x10+x20)/2, (y10+y20)12).   (8)

In this display mode, the distance |r10-r20|, measured in the planeΠ(φ), is precisely the present separation distance for the first andsecond AVs.

Optionally, the present velocity vectors v10=(vx10,vy10,vz10) andv20=(vx20,vy20,vz20) can be shown “anchored” at the respectivelocations, r10 and r20. Generally, these present velocity vectors willnot lie in the plane Π(φ) unless vy1/vx1=tan φ and/or vy2/vx2=tan φ.

A third display mode, illustrated in FIGS. 3A and 3B, provides a nadir(overhead) view of the first and second AVs. The anchor point for thishorizontal display plane Π(3), with unit length normal vector n=k,preferably passes through the present location r10 of the first AV(z=z10), or through the present location r20 of the second AV (z=z20),or through a location intermediate between the present vertical heightsof the first and second AVs (z=z120=fz10+(1−f)·z20, with 0≦f≦1 (forexample, f=0.5). The present locations, r10 and r20, of the first andsecond AVs are projected vertically onto the plane Π(3), using Eq. (3)with n=k, with arrows that point that toward each other if CRV<0 andthat point away from each other if CRV>0. This is preferably shown in aperspective view.

In a fourth display mode, an initial separation vectorΔr12(t0)=r1(t0)−r2(t0) and a separation distance squared,d(t;1,2)²=|Δr12(t)|² are determined for first and second AVs, usingobserved or estimated vector values for the present location rj(t0)(j=1, 2), the present velocity vector vj0=vj(t0) and the presentacceleration vector aj0=aj(t0) for each AV. A prediction of a futureseparation vector is estimated, using r10-r20, v10-v20, and a10-a20, andan analysis disclosed in a preceding patent application, U.S. Ser. No.11/888,070 (incorporated by reference herein), and one or more times,t=t(min), are determined for which d(t;1,2)²=|Δr12(t)|² is minimized.One or three real solutions, t=t(min) can be found, and interest centerson a first real solution for which t(min)≧t0.

A unit length normal n(4)=(v10̂a10)/|v10̂a10| is determined, and a displayplane Π(4) is identified, with normal vector n(4) and anchor point givenby the location r10. Where the vectors v10 and a10 are substantiallyparallel, a vertically oriented plane Π′(4), generated by the vectors kand v10, becomes the replaces the plane Π(4), with corresponding normalvector n′(4)={k̂v10}/|k̂v10|. The location r20 of the second AV is shownrelative to the display plane Π(4), or is projected onto Π(4), using theprojection formalism disclosed in Appendix A or an equivalent formalism.A predicted AV trajectory r1(t) (t0(fixed)≦t≦t(min)) for the first AVlies in, and does not deviate from, the display plane Π(4), and one ormore intermediate locations r1(t′) for the first AV trajectory are shownon the display plane Π(4). The system optionally indicates the minimumseparation distance d(t=t(min);1,2) and determines whether this minimumseparation distance satisfies d(t=t(min);1,2)≦r(thr0). Here, r(thr0) isa selected conflict radius of a sphere Sph(4), centered at the locationr1(t), and the separation vector Δr12(t) should avoid the sphereinterior. The interior of Sph(4) may be a region where a collision ofAV1 and AV2 may occur, or where these two AVs may experience a nearmiss, for example, r(thr0)=100-1320 feet.

Where d(t=t(min);1,2)>r(thr0), the separation distance d(t;1,2) for thepredicted trajectories of AVs number 1 and 2 will always be greater thanr(thr0). In this situation, no further graphics are required for thefourth display mode. Where d(t(min);1,2)≦r(thr0), the system (1)estimates a first time, t=t1, that satisfies t0≦t1≦t(min) andd(t=t1;1,2)=r(thr0), (2) determines a location r1(t1), (3) displays acircle Cir(4) in the display plane Π(4), centered at r1(t1), of radiusr(thr0) (representing a predicted conflict sphere), (4) determines aremaining time, Δt(rem)=t1-t0, before conflict will occur, and (5)provides a visually perceptible and/or audibly perceptible signalindicating that a conflict will occur and the remaining time Δt(rem).This situation is illustrated in FIG. 4. Optionally, the systemrecommends an acceleration increment, Δa1(t0) that AV1 should execute inorder to avoid the predicted conflict.

In the fourth display mode, a predicted separation vector Δr12(t;1,2) isdetermined by the relation

Δr12(t)=r10−r20+{v10−v20}(t−t0)+(½){a10−a20}(t−t0)² (t≧t0),   (9A)

rj0=rj(t0) (j=1, 2),   (9B)

vj0=vj(t0) (j=1, 2),   (9C)

aj0=aj(t0) (j=1, 2).   (9D)

This minimum value is a real solution, t=t(min), of a cubic equation

$\begin{matrix}{{\left. {{\left( {{\partial\text{/}}{\partial t}} \right){d\left( {t;1;2} \right)}^{2}} = {{2\left( {{r\; 10} - {r\; 20}} \right)\left( {{v\; 10} - {v\; 20}} \right)} + {2\left\{ {\left\{ {{v\; 10} - {v\; 20}} \right\}^{2} + {2\left\{ {{r\; 10} - {r\; 20}} \right\} \left\{ {{a\; 10} - {a\; 20}} \right\}}} \right\} \left( {{t\left( \min \right)} - {t\; 0}} \right)} + {6\left\{ {{v\; 10} - {v\; 20}} \right\} \left\{ {{a\; 10} - {a\; 20}} \right\}}}} \right\} \left( {{t\left( \min \right)} - {t\; 0}} \right)^{2}} + {4\left\{ {{a\; 10} - {a\; 20}} \right\}^{2}\left( {{{t\left( \min \right)} - {t(0)}^{3}} = 0.} \right.}} & (10)\end{matrix}$

Eq. (10) has one or three real solutions. If no real solution, t=t(min),exists for which t(min)>t0, the time point, t=t(min), of closestapproach for the two AVs has already passed (i.e., t(min)<t0), and nosubsequent action can be taken that will affect the minimum separationdistance.

A sequence of two or more locations for each of the first and secondAVs, r1(t) and r2(t), beginning at t=t0, is optionally displayed in thisfourth mode, representing separate trajectories for each of the AVs.Assuming that t(min)>t0 in this fourth display mode, the distancesquared of closest approach d(t(min);1;2)² will occur at some time inthe future (t=t(min)>t0), and the system determines whether

d(t(min);1,2)≦r(thr0),   (11)

where r(thr0) is the conflict radius. Optionally, the first conflicttime, t=t1(t0≦t1t(min))), at which the separation distance|Δr12(t)|≦r(thr0), is determined by

$\begin{matrix}{{{{\left. {{{\Delta \; r\; 12\left( {t = {t\; 1}} \right)}}^{2} = {\left( {{r\; 10} - {r\; 20}} \right)^{2} + {2\left( {{r\; 10} - {r\; 20}} \right)\left( {{v\; 10} - {v\; 20}} \right)} + {\left\{ {\left( {{v\; 10} - {v\mspace{11mu} 20}} \right\}^{2} + {2{\left( {{r\; 10} - {r\; 20}} \right\} \cdot \left\{ {{a\; 10} - {a\; 20}} \right\}}}} \right\} \left( {{t\; 1} - {t\; 0}} \right)^{2}} + {2{\left\{ {{v\; 10} - {v\; 20}} \right\} \cdot \left\{ {{a\; 10} - {a\; 20}} \right\}}}}} \right\} \left( {{t\; 1} - {t\; 0}} \right)^{3}} + {\left\{ {{a\; 10} - {a\; 20}} \right\}^{2}\left( {{t\; 1} - {t\; 0}} \right)^{4}}} = {r\left( {{thr}\; 0} \right)}^{2}},} & (12)\end{matrix}$

and the controller is made visually aware and/or audibly aware of howclose in time, Δt=t1-t0, is the (first) conflict point. An accelerationincrement, Δa1 or Δa2, is recommended, visually and/or audibly, foravoiding the conflict.

Optionally, the fourth display mode is presented whenever either of twosituations occurs, illustrated in FIG. 5A and FIG. 5B, using avertically-oriented cylinder or some other suitable geometric regionrepresenting a user-defined volume of airspace. In a first situation forthe fourth display mode, illustrated in FIG. 5A, the system determinesthat the present separation distance |Δr12(t0)| satisfies

|Δr12(t0)|=|r10−r20|≦r(thr1),   (13)

where r(thr1) is a first threshold radius, greater than r(thr0), forpotential conflict. In this first situation, the fourth display mode ispresented if the present separation distance |Δr12(t0)| is no greaterthan a first selected positive potential conflict radius r(thr1)(>t(thr0)),

In a second situation for the fourth display mode, illustrated in FIG.5B, the system determines that an estimated future separation distance|Δr12(t0+Δt;est)|, defined by

|Δr12(t0+Δt;est)|² =|r10−r20+{Δt(∂/∂t) (r1(t)−r2(t))_(t=t0)}|²≦r(thr2)²,   (14)

is no greater than a second selected positive potential conflict radiusr(thr2) (>r(thr0)), where Δt is a selected positive time value (e.g.,Δt=15-60 sec). In this second alternative, the fourth display mode ispresented if the estimated future separation distance, for a selectedfuture time t=t0+Δt, is no greater than r(thr2) that is greater than theconflict radius r(thr0). Optionally, in the first alternative and/or thesecond alternative, the locations of AVs 1 and 2 can be displayed withina chosen geometric shape, such as a cylinder CR of suitable height anddiameter. Optionally, the fourth display mode can also be presentedwhere neither the first situation nor the second situation occurs. Asphere of radius r(thr0) or r(thr1) or r(thr2) is somewhat analogous toa “tau area” in TCAS.

In a fifth display mode, illustrated in FIG. 6, a display plane Π(5) isdefined by the present velocity vectors, v10 and v20, of the first andsecond AVs, with unit length normal vector n(5) for this plane havingthe components

n(5)=(v10̂v20}/|v10̂v20|.   (15)

This assumes that v10 and v20 are not substantially parallel. Thedisplay plane Π(5) contains the present location r10, and contains aprojection of the second AV r20, parallel to the normal vector n(5),onto the plane Π(5), computed in Appendix A according to

r20(proj)=r20−{(r20−r20)·n(5)}n(5)   (16)

The display plane Π(5) is defined by the normal vector n(5) in Eq. (15),and an anchor point r10 that is the present location of the first AV.The second AV present location r20 is projected onto the display planeΠ(5) as indicated in Eq. (16). The magnitude of the projected distance

Δr12(proj)=r10−r20(proj),   (17)

measured in the plane Π(5), is an visual estimate or lower bound of thepresent separation distance d(t0;1,2)=|r10−r20|.

As indicated in Appendix A, the unit length normal vector n(5) isexpressible in terms of direction cosines,

n(5)=(cos α1, cos α2, cos α3),   (18)

cos²α1+cos²α2+cos²α3=1,   (19)

the plane Π(5) may be expressed in coordinates as

(x−x0)cos α1+(y−y0)cos α2+(z−z0)cos α3=0,   (20)

where (x0, y0, z0) are the coordinates of an anchor point, for example,

(x0, y0, z0)=r1(t0) or r2(t0).   (21)

In a sixth display mode, illustrated in FIG. 7, the display plane Π(6)is defined by the present vectors, v10 and a10, with unit length normalvector

n(6)=(v10̂a10}/|v10̂a10|,   (22)

and contains the present location r10, as an anchor point. This assumesthat v10 and a10 are not substantially parallel. The equation for thedisplay plane is determined as in Appendix A, Eq. (A3), using thedirection cosine values determined for the normal vector n(6) and theanchor point coordinates for the present location r10:

r10=(x10,y10,z10)=(x0,y0,z0).   (23)

The present location r20 of the second AV is projected parallel to thenormal vector n(6) onto the plane Π(6) and displayed as r2(t0;proj), asin Eq. (16). Where v10 and a10 are substantially parallel, a verticallyoriented plane, generated by the vectors k and v10, becomes a planeΠ′(6), replacing the plane Π(6), with corresponding normal vectorn′(6)={k̂v10}/|k̂v10| Trajectories, r1(t) and r2(t) are shown for thefirst and second AVs from the present time value, t=t0, to a time,t=t(min), corresponding to closest approach for the AVs, which willgenerally not lie on the plane Π(6).

Each of the display screens in each of the display modes (1-6)optionally includes a supplemental first scale S1 that graphicallyprovides (1) a visually perceptible first length L1 that is linearlyproportional to a present separation distance |r10-r20| of the first andsecond AVs and optionally provides a supplemental second scale S2 thatgraphically provides a second length L2 that is linearly proportional toa closing rate value, CRV=(∂/∂t)|r1(t)−r2(t)| at a chosen value of timet, such as the present time, t=t0 and indicate with two opposed arrowswhether the CRV>0 (arrows point away from each other) or CRV<0 (arrowspoint toward each other).

Each of the first, second, third, fourth, fifth and sixth display modesis referenced to a display plane, where the normal vector defining thisplane is, or is proportional to, one of the vectors

n=i, j, k, −i si φ+j cos φ, (v10̂v20)/|v10̂v20|, or (v10̂a10)/|v10̂a10|,  (24)

The anchor point(s), if any, is one of the locations

AP=r10, r20, f z10+(1−f) z20, r1(t(min)), or r2(t(min)).   (25)

A projection r(proj) of a vector r onto a display plane having a unitlength normal vector n is determined as

r(proj)=r−{(r·r(i))·n}n,   (26)

where r(i) is the location of an intersection of the vector r with thedisplay plane.

The preceding development has illustrated six different display modesfor two or more AVs and has considered the possibility of conflict,according to which the distance of closest approach for these AVsbecomes no greater than a conflict radius r(thr0). One or more of theseAVs may be an unmanned aerial vehicle that is remotely controlled by aground-based or airborne-based UAV controller, with one or more displaymodes being presented to the controller at a sequence of spaced aparttimes (e.g., with spacing Δt=2-15 seconds, or longer). Several or all ofthese display modes may be presented simultaneously, or sequentially, toa remotely located controller of a UAV or to a cockpit-based pilot of amanned aircraft. In a first alternative, each of the AVs may be capableof independent flight. In a second alternative, the recommendationsassociated with the display modes may be transferred to an autopilotsystem archive.

The display modes disclosed here are preferably implemented by acomputer that is programmed to receive and store: (i) the presentlocation components, r1(t0) and r2(t0); (ii) the present velocity vectorcomponents, v1(t0) and v2(t0); (iii) the present acceleration vectorcomponents, a1(t0) and a2(t0); (iv) the conflict radius and potentialconflict radii, r(thr0), r(thr 1) and r(thr2); and (v) to compute otherscalars and vector components as needed.

The different display modes are not intended to replace the screen viewspresented to an air traffic controller. The different display modes ofthis invention are intended to be used by any AV pilot including UAVcontrollers. The different display modes represent a specific AV'sflight safety situation symbolically in two dimensions, with eachdisplay mode having its characteristic strengths and deficits. It islikely that a given UAV controller or AV pilot will develop a humanfactors driven preference for one, two or three of these modes.Therefore, each of the display modes is made available to suit thepreferences of a given UAV controller or AV pilot.

Optionally, the UAV controller or AV pilot has a larger or centralizedprimary screen to display a chosen mode and one, two or more secondaryscreens to display modes that may complement information shown on theprimary screen. In FIGS. 11, and 12 and 13, modes 1 and 2 and 4 aredisplayed as primary, respectively, and other modes are displayed assecondary, respectively. This treatment can be extended to provision ofmany screens, one being primary and all others being secondary. Thecontroller presses one or more of a group of M buttons to bring up aspecified display mode as primary, and may further specify one or moreof the M-1 other display modes as secondary.

The information presented by the audiovisual display modes might beequated with an Air Traffic Control perspective; however, this is notthe case. The fundamental difference is that the computations andalgorithms of the sense-and-avoid display system focus only on potentialairborne conflicts involving a selected AV (e.g., the UAV being operatedby a ground-based controller). The system processes 3D data from a datasource, such as ground-based radar, and then evaluates, identifies,prioritizes, and declares action for potential conflicts with other AVsin a timely manner. In this AV-centric framework, the system providesthe UAV controller with a flight safety capability of maintaining a safeseparation distance from other AVs during UAV flights that extend beyondthe visual range of ground-based observers and airborne observers inchase planes.

When compared with the forward-looking perspective of pilots in mannedaircraft, the difference is that the sense-and-avoid display system iscapable of providing UAV controllers with information from a 360° 3Dvolume of airspace surrounding their respective UAV. This capability ismade possible using 3D data sources. An example of such a 3D data sourceis the Sentinel radar manufactured by Thales Raytheon Systems that hasbeen integrated with the herein described sense-and-avoid displaysystem. The Sentinel radar detects the x,y,z positions of cooperativeAVs equipped with an identification device (e.g., a transponder) as wellas noncooperative AVs not equipped with an identification device.

In the operational framework of flight safety, the invention describedherein applies to an Unmanned Aircraft System (UAS) defined as includingan Unmanned Aerial Vehicle (UAV), a ground control station, a UAVcontroller, and any associated equipment, software and communicationlinks that support UAV flight operations.

Appendix A. Display Plane Geometry.

Simple analytical geometry techniques can be used to relate a unitlength normal vector components of a display plane and coordinates of ananchor point on the plane to components of an equation defining thatplane. If the unit length normal vector n for the plane is expressiblein terms of direction cosines,

n(5)=(cos α1, cos α2, cos α3),   (A1)

cos²α1+cos²α2+cos²α3=1,   (A2)

the plane Π may be expressed in coordinates as

(x−x0)cos α1+(y−y0)cos α2+(z−z0)cos α3=0,   A3)

where (x0, y0, z0) are the coordinates of an anchor point, asillustrated in FIG. 8. In the context presented here, the anchor pointusually coincides with the present location (vector) of the first orsecond AV:

(x0, y0, z0)=r1(t0) or r2(t0).   (A4)

A projection of a vector, such as r, onto a plane that is defined by aunit length normal vector n can be expressed in vector notation as

r(proj)=r−{(r·r(i))·n}n.   (A5)

as illustrated in FIG. 9, where r(i) is an intersect vector, drawn tothe location where r intersects the plane Π. The intersect location r(i)is determined as follows. Express the vector r in parametric form as

x−x1=e·s, y−y1=f·, z−z1=g·s, (0≦s≦|r|)   (A6)

e ² +f ² +g ²=1,   (A7)

where s=0 corresponds to the origin of r and s=|r| corresponds to theother end of the vector r, the location r. In this configuration,x1=y1=z1=0. The unit length normal vector n has components n=(a,b,c)with a²+b²+c²=1, and a plane Π with the normal vector n can be expressedparametrically as

a(x−x0)+b(y−y0)+c(z−z0)=0,   (A8)

where (x0, y0, z0) is an anchor point of this plane. The intersectvector r(i) corresponds to a location on the plane Π for which

$\begin{matrix}{{{{a\left( {{s_{I}^{*}e} - {x\; 0}} \right)} + {b\left( {{s_{I}^{*}f} - {y\; 0}} \right)} - {c\left( {{s_{I}^{*}g} - {z\; 0}} \right)}} = 0},} & ({A9}) \\\begin{matrix}{s = {s_{I} = {\left\{ {{a\; x\; 0} + {{by}\; 0} + {{cz}\; 0}} \right\} \text{/}\left\{ {{a \cdot e} + {b \cdot f} + {c \cdot g}} \right\}}}} \\{{= {\left\{ {{{ax}\; 0} + {{by}\; 0} + {{cz}\; 0}} \right\} \text{/}\left( {n \cdot r^{\bigwedge}} \right)}},}\end{matrix} & ({A10}) \\{{r^{\bigwedge} = {r/{r}}},} & ({A11}) \\{{r = {r^{\bigwedge}s_{I}}},} & ({A12})\end{matrix}$

where the line parameter s_(I) may have any positive or negative or zerovalue. The location r(i) is the intersection with the plane Π of a linesegment LL (of undetermined length), aligned with the vector r. Thesegeometric quantities are illustrated in FIG. 9.

Appendix B. Determination of Acceleration Increment to Avoid Conflict.

Assume AV2 is approaching AV1 with a velocity vector v2(t) andacceleration vector a2(t), as determined and made available to the firstAV controller, where a2(t) is assumed to be substantially constant, asillustrated in FIG. 10. Assuming that v2(t) and a2(t) are notsubstantially parallel, the cross product, v2(t)̂a2(t) characterizes aunit length normal vector

n2={v2(t)̂a2(t)}/|v2(t)̂a2(t)|  (B1)

that is perpendicular to an instantaneous approach plane Π′ thatcontains v2(t) and a2(t) and has the location r2(t) as an anchor point.Note that n2 need not be entirely vertical or entirely horizontal. Oneoptimal maneuver for AV1 is to move perpendicular to the plane Π′, awayfrom this plane. The approach plane Π′ may be defined by coordinates(x,y,z) satisfying

n2·r−L=0,   (B2)

r=(x,y,z),   (B3)

where L is a perpendicular distance from the origin to the plane Π′. Theacceleration increment Δa1 is preferably chosen to be parallel (oranti-parallel) to the normal vector n2, which is defined by the presentvectors v20 and a20 of AV2. Identification of the normal n2 for theapproach plane Π′with sufficient accuracy will often require use of adirectional antenna with very good angular resolution, in order toaccurately determine or estimate the vectors v20 and a20.

In an alternative response to potentially entering a conflict volume,AV1 can execute a substantial deceleration Δa1 in its present heading(reduced magnitude, same direction), chosen so that AV1 will reach andpass through its location r1(t1) long after AV2 has reached and passedthrough its corresponding location r2(t1), to ensure that the separationdistance d(t;1,2) is always greater than r(thr0). This assumes that AV2does not significantly alter its own velocity, acceleration ordeceleration vectors.

What is claimed is:
 1. A method for displaying present and predictedfuture locations, velocities and accelerations of each of at least firstand second aerial vehicles (AVs), the method comprising providing acomputer that is programmed: to receive or otherwise provide estimatesof present locations rj (t0) of a selected number J of AVs, numberedj=1, . . . , J (J≧2), for a present time, t=t0, where the locationsrj(t0) are determined with reference to a Cartesian coordinate system(x,y,z) having unit length vectors i, j and k oriented parallel to x-,y- and z-coordinate axes, with initial vector location valuesrj(t0)=(xj0,yj0,zj0); to provide estimates of at least one of presentvelocity vectors vj (t0) and present acceleration vectors aj(t0) for theJ AVs, for the present time, t=t0; to provide an estimate of locationvectors rj(t), velocity vectors vj(t) and acceleration vectors aj(t) forat least two spaced apart times t that are greater than t0; to providean estimate of a time, t=t(min)≧t0, for which a square of a distanceseparation value d(t;1,2)²=|Δr12(t)|²=|r1(t)−r2(t)|² attains a locallyminimum value for the AVs j=1 and j=2, and to estimate the locationsrj(t=t(min)) for j=1 and j=2; to choose said value t=t(min) to be avalue of time that is greater than said value t=t0 and that satisfies(∂/∂ t)d(t = t(min ); 1; 2)² = 2(r 10 − r 20) ⋅ (v 10 − v 20) + 2{{v 10 − v 20}² + 2{r 10 − r 20} ⋅ {a 10 − a 20}}(t(min ) − t 0) + 6{v 10 − v 20} ⋅ {a 10 − a 20}}(t(min ) − t 0)² + 4{a 10 − a 20}² − (t(min ) − t 0)³ = 0,where d(t;1;2)²=|Δr12(t)|2=|r1(t)−r2(t)|² is a distance of separationsquared between the locations r1(t) and r2(t); to provide a unit lengthnormal vector n={v10̂a10}/|v10̂a10|, of a display plane for the J AVs,where v10 and a10 are not approximately parallel to each other, and tochoose an anchor point AP for the display plane Π to be at least one ofthe locations r10 and r20; to visually represent, on a display screen, adisplay plane Π defined by the normal vector n and passing through atleast one display plane anchor point AP; to visually represent, on thedisplay screen, the locations r1(t0) and r2(t0) on or adjacent to thedisplay plane Π, determined with reference to the normal vector n. todetermine whether the separation value d(t(min);1,2) satisfiesd(t(min);1,2)≦r(thr0), where r(thr0) is a selected positive conflictradius; and when d(t(min);1,2)≦r(thr0), to estimate a first value, t=t1,in time that satisfies t0≦t1≦t(min) and d(t1;1,2)=r(thr0), to estimateand to display a remaining time Δt(rem)=t1-t0 as at least one of avisually perceptible signal and an audibly perceptible signal, and toestimate a predicted location r1(t=t1), to display a circle on thedisplay plane Π, centered at the location r1(t=t1) and having a circleradius r(thr0); and to display a location r1(t′) for at least two valuesof time t′ in a range t0≦t′≦t1; and
 2. The method of claim 1, whereinsaid computer is further programmed to provide a recommendedacceleration increment Δa1, to be added to said acceleration vector a10,that may allow said AV1 and said AV2 to avoid a conditiond(t;1,2)≦r(thr0) for any value of time t in the range t0≦t<t(min). 3.The method of claim 2, wherein said computer is further programmed: todetermine a normal vector n2 for an approach plane Π′ by a relationn2={v20̂a20}/|v20̂a20|, where v20 and a20 are not approximately parallelto each other; and to recommend said acceleration increment Δa2 for saidAV1 to be approximately parallel or approximately anti-parallel to theapproach plane normal vector n2.
 4. The method of claim 2, wherein saidcomputer is further programmed: to recommend, as said accelerationincrement Δa1 for said AV1, a reduction of a magnitude of said velocityv10, while maintaining a direction of said velocity v10 approximatelyunchanged.
 5. The method of claim 1, wherein said computer is furtherprogrammed: to estimate said time t=t1 at which said separation distancesatisfies d(t=t1;1,2)=r(thr0), according to a real valued solution,t1-t0, of an equationΔ r 12(t = t 1)² = (r 10 − r 20)² + 2(r 10 − r 20) ⋅ (v 10 − v 20)(t 1 − t 0) + {(v 10 − v 20}² + 2(r 10 − r 20} ⋅ {a 10 − a 20}}(t 1 − t 0)² + 2{v 10 − v 20} ⋅ {a 10 − a 20}}(t 1 − t 0)³ + {a 10 − a 20}²(t 1 − t 0)⁴ = r(thr 0)².6. The method of claim 1, wherein said computer is programmed to executesaid operations in claim 1 only when a present separation distanced(t0;1,2) is less than a selected positive first potential conflictradius r(thr1) that is greater than said radius r(thr0).
 7. The methodof claim 1, wherein said computer is further programmed to execute saidoperations in claim 1 only when an estimated future separation distance,defined by d(1,2;t0+Δtest)²=|r10-r20+{Δt(∂/∂t) (r1(t)−r2(t))(t=t0)}|²,is less than a selected second potential conflict radius r(thr2) that isgreater than said radius r(thr0), where Δt is a selected positive timevalue.
 8. The method of claim 1, further comprising: when at least onetime value, t=t1, in a range t0≦t≦t(min) exists for whichd(t1;1,2)≦r(th0), providing at least one of an audio indication and avisual indication that advises said controller that said first andsecond AVs are predicted to move within a separation distance r(thr0) ofeach other.